Effective Use of Likert Scales in Visualization Evaluations: A Systematic Review

Likert scales are often used in visualization evaluations to produce quantitative estimates of subjective attributes, such as ease use or aesthetic appeal. However, the methods collect, analyze, and visualize data collected with inconsistent among papers. In this paper, we examine a tool for measuring response systematic review 134 published between 2009 2019. We find that papers both objective measures do not hold same reporting analysis standards aspects their evaluation, producing less rigorous work qualities measured by scales. Additionally, demonstrate many interpretations discrete continuous may even sacrifice statistical power applying nonparametric tests unnecessarily. Finally, identify instances where key details about item construction potential bias participant responses omitted from evaluation methodology reporting, inhibiting feasibility reliability future replication studies. summarize recommendations other fields best practices evaluations, based on results our survey. A full copy paper all supplementary material available at https://osf.io/exbz8/.